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Brian Lemay, Ph.D, Independent criminal law researcher, US 


Abstract: Nowadays the phenomenon of artificial intelligence is 
a term that was created and used initially by other sciences. The 
term has passed indirectly via various steps towards a legal 
system which is full of gaps both at national and European level. 
New technologies have been a topic of discussion for years not 
only for jurists but also for other categories of professionals. 
Unfortunately, the European reality that interests us in this paper 
has shown that the time has now come to try to regulate what is 
called artificial intelligence as well as the role it plays in our 
society. Our paper focuses on the criminal liability of the person 
who deals with an algorithmic machine. However, is it a 
criminal liability that falls only on the person or does it also 
involve the machine? The questions are many, and the European 
path in this sector is still obscure and on primitive steps. Surely 
in the near future we would talk more precisely about the 


criminal liability that covers this sector. 
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Introduction 
Since evidence collected in a technological and digital way in 
international and national criminal trials, we have started talking 
about artificial intelligence (AI) (Freitas et al., 2014; Hallevy, 
2015; Gless, Silverman, Weigend, 2016; Miiller, Bostrom, 2016; 
Seher, 2016; Kowert, 2017; Simmler, Markwalder, 2017; 
Pagallo, Quattrocolo, 2018; Abbott, Sarch, 2019; Burchard, 
2019; Palace, 2019; Hu, 2019; Lagioia, Sartor, 2020; 
Nersessian, Mancha, 2021; Kovac, 2022; Giinsberg, 2022)'. The 
diffusion of this system of new technologies as well as of the AI 
that function in our daily life in an autonomous and unexpected 
way remains questionable both because the veracity of these 
evidences are difficult to demonstrate but also because we do 
not know if it exists and if yes what is the relative responsibility 
that machines have. In other words, these are doubts that enter 
the national, European, international criminal sphere and seek an 
answer (Pagallo, 2018). 

1European Commission, High-Level Expert Group on Artificial Intelligence, A 


definition of AI: Main capabilities and scientific disciplines, 2018: 
https://ec.europa.eu/futurium/en/system/files/ged/ai_hleg_ definition_of_ai_18 decem 


ber_1.pdf. 
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With the expression artificial intelligence we speak: 


“(...) of machines equipped with a very significant calculation capacity. 
Artificial intelligence systems are, in fact, computers (and computer 
programs) that combine large amounts of data (so-called Big Data), with the 
aim of learning how to manage future decision-making processes (...). These 
are not machines capable of making real decisions, but more simply of 
drawing certain conclusions from the data that can be deduced from previous 
experiences (...)’(Gillespie, 2014). 

Already at the European level we have the European Parliament 


which, through the Resolution of October 2021 “Artificial 
intelligence in criminal law” put the first ideas on the subject by 
stating: 


“(...) a clear and fair regime for attributing legal responsibility and 
imputability of the potential negative consequences produced by these 
advanced digital technologies (...)’”. 

No attention was paid to the related criminal liability as the 


Resolution did not provide instructions but concentrated 
generally on the European dimension. The various questions due 
to the continuous technological progress and the relationship 
with the criminal law have started to play an important role 
since we have been told about a shock of modernity (Wells, 
2011). In our opinion we can speak of a particular type of 
liability for damage caused by a product that was defective in 
the area of environmental crimes. It is a not very developed area 
but criminal law was always a tool to punish, to compensate and 
the difference in national or European criminal law always 
2European Parliament resolution of 2021 on artificial intelligence in criminal 
law and its use by police and judicial authorities in criminal matters 


(2020/2016(IND), 13: https://www.europarl.europa.eu/doceo/document/TA-9-2021- 
0405_EN. html 
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remains a topic under discussion and also difficult to arrive at 
concrete conclusive results (Abbott, Sarch, 2019). 


According to Abbott and Sarch, 


“AT systems raise an irreducibility problem for criminal law (the so-called 
“irreducibility challenge”) as it could be extremely difficult, if not 
impossible, to reduce a crime committed by an AI system to the action of an 
individual human being (Santoni De Sio, Mecacci, 2021). Four 
characteristics are possessed by AI systems: autonomy, i.e. the ability to 
cause a harmful event without the system being directed to do so by a human 
agent; opacity, i.e. the impossibility-especially when dealing with more 
advanced systems such as those based on deep learning (Hao, 2018) - of 
being able to obtain an explanation on how the system, starting from the 
input (A), obtained the harmful output (B); complexity, i.e. the fact that the 
creation of an AI system is often the result of the contribution of many 
individuals, developed over a long period of time, as well as the fact that the 
system may have been trained on databases heterogeneous and open source; 
and finally unpredictability, i.e. the ability for the system to undertake 
activities not foreseen by its original programming (...)” (Abbott, Sarch, 
2019). 

After these first statements it is noted that the criminal 


attribution of responsibility towards man/men is open to an 
intelligent system. Among the first questions is the omissive 
conduct that has to do with a juridical duty and a fact that 
presents itself as an impediment to an event. 

On the other hand, human responsibility needs a position of 
guarantee, as well as the existence of a precautionary rule as a 
consequence of a positive attitude or one based on the 
experience and configurability of a model agent. Furthermore, 
does the knowability of a rule also mean the predictability of a 
criminal event or not? We need the enforceability of a behavior 


that conforms to the human being. This contribution has peculiar 
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characteristics that interacts with the AI both on the subject of 
responsibility and also on the title of guilt. 
Involvement with an AI system as we understand it is very 
complex due to the paucity of scientific knowledge of this type 
of technology as well as the lack of a foundation of conventions 
on the subject. There are no ad hoc rules that can be known by 
the human agent, just as the machine must exercise the relative 
control since it is built, organized to prevail. The harmful event 
should be prevented and is unpredictable. 
First of all, an analysis of the accountability is needed (Novelli, 
Taddeo, Floridi, 2023) which approaches the theme of the AI- 
regulation after the notion of the impact area of the AI and its 
relationship with criminal matters. Of course, this does not 
outline the line of an area-operation circle that is carried out in 
assessments on the extent of the impact of the AI in a specific 
sector of criminal law. For once again the European Parliament 
of October 2021 entitled: “Artificial intelligence in criminal 
law” and the related proposal for a European regulation known 
as the AI Act? are steps forward on the notion of human 
surveillance (“human oversight”)* which opens the debate of the 
identification of the relationship between this notion and 
3European Commission, Proposal for a regulation of the European parliament 
and of the council establishing harmonized rules on artificial intelligence (law on 
artificial intelligence) and amending certain legislative acts of the union, 
COM/2021/206, 21 April 2021: https://eur-lex.europa.eu/legal-content/EN/TXT/? 


uri=celex%3A52021PC0206 
Ahttps://data.consilium.europa.eu/doc/document/ST-11124-2022-INIT/en/pdf 
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criminal guilt. 

It is obvious that more from national criminal law we focus on 
European law. Over time, a new discussion impact is quickly 
acquired in the European context that approaches towards the 
regulation of artificial intelligence and the creation of the “good 
AI society” (Floridi and others, 2018; Roberts, 2021), including 
also such sector in the work of the European institutions, given 
that the continuous evolution of AI systems (Ravidran, 2020) 
are the basis for the objectives of the European criminal policy. 
Within this context, the Europol Innovation Lab has published a 
report on crimes committed using deepfake technologies (Thi 
Nguyen, T. et al., 2019) proposing: 


“(...) the emergence of new forms of crime for the next decade, which will 
bring with them many difficulties from the point of view of the attribution of 
criminal responsibility (...)”. 

Continuing with the project “Accountability Principles for 


Artificial Intelligence” (AP4AI)°, which brings together 
Eurojust, Europol, CEPOL as well as the research centre: Center 
of Excellence in Terrorism, Resilience, Intelligence and 
Organized Crime Research (CENTRIC)’ as a development basis 
of a chain of creation of principles that guarantee the ethical, 


transparent and responsible utility of AI systems especially in 


5Europol (2022), Facing reality? Law enforcement and the challenge of deepfakes, 
an observatory report from the Europol Innovation Lab, Publications Office of the 
European Union, Luxembourg, 16. 

6https://www.ap4ai.eu/ 

7https://www.netlaw.bg/en/a/centric-centre-of-excellence-in-terrorism-resilience- 
intelligence-and-organised-crime-research-sheffield-hallam-university-united- 
kingdom# 
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law enforcement agencies. 


Accountability, responsibility and liability, Are they new 
penal concepts? 

When we talk about accountability in the AI sector we mean its 
regulation and development in an ethical way through the use of 
soft law principles rather than its elaboration by relative hard 
law tools®. 

The notion of accountability has been referred to in the Artificial 
Intelligence High-Level Expert Group (AI HLEG) which was 
established by the European Commission in the document 
entitled: “Ethical Guidelines” which: 


“(...) represents a recurring element in initiatives of European regulation 
(Barrio Andrés, 2021; Kerrigan, 2022)°, as well as in the broader global 
panorama of the so-called AI ethics (...)” (Jobim and others, 2019; Thi 
Nguyen, 2019; Fjeld, 2020)!9. 

The doctrine in this regard characterizes accountability as a 


sufficient ethical tool to regulate the AI. Its is represented by the 
example of an ethics that includes a “juridical conception” 


(Anscombe, 1968), such as: 


8President of the Republic of 14 April 2021 (Ordonnance n° 2021-443 du 14 
avril 2021 relative au régime de responsabilité pénale applicable en cas de 
circulation d'un véhicule a délégation de conduite et a ses conditions d'utilisation, 
TRAT2034523R). 


QResolution of the European Parliament, Framework on the ethical aspects of artificial 
intelligence, robotics and related technologies, 2020/2012(INL), 20 October 2020, artt. 9, 
22, 23, 72, 96, 102: https://www.europarl.europa.cu/doceo/document/TA-9-2020- 
0275_EN.html 

10See for example: OCSE, Recommendation of the Council on Artificial 
Intelligence, OECD/LEGAL/0449, 2019). UNESCO, Recommendation on the ethics 
of artificial intelligence, SHS/BIO/REC-AIETHICS/2021, 2021). 
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“(...) a sort of draft of the law (...) where ethics is not “toothless” at all but is 
simply used in the wrong way (...)” (Rességuier, Rodrigues, 2020; Munn, 
2022). 


If so, what would be the meaning of the word accountability and 
what kind of relationship or link does it have with criminal 
liability? 

Accountability does not mean responsibility of liability despite 
the fact that it is a notion that is connected with many notions of 
responsibility. Liability includes the willful or unintentional 
actions either of a person or of others. As far as the actions of 
others are concerned, the legal situation results in the violation 
of an obligation and at the same time also in an unlawful act, 1.e. 
an obligation, the ability to report. Thus accountability does not 
include responsibility but has the duty to suffer the relative 
consequences, i.e. it is an attitude that uses the capacity of an 
obligation that takes into consideration a social system. The 
boundaries between accountability, responsibility and liability!! 
are difficult to be distinguished. Liability has to do with 
qualified conduct, accountability and responsibility have a 
broader notion that also contains rules of a moral and social 
nature as well as standards of courtesy. 

Accountability is defined as a notion of a procedural nature that 
symbolizes the accused or who being placed in the dock typical 


elements of European criminal trials and also of the common 


11https://commission.europa.eu/business-economy-euro/doing-business-eu/ 
contract-rules/digital-contracts/liability-rules-artificial-intelligence_en 
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law given that they appear before a universal community (Kool, 


2014; Ausserladscheider Jonas, 2021): 


“(...) Paccountability is the process aimed at a subsequent public assessment 
of a person’s conduct in a given case in order to evaluate whether this 
conduct was required and/or justified by this person’s responsibility and, 
once this evaluation is executed, to establish who can be held liable for all 
consequences of the conduct (...)” (Giesen, Kristen, 2014). 


According to Duff: 


“(...) responsibility is a “relational” concept, since it concerns the relationship 
that is established between a responsible person (A), an object (X) for which 
this person is responsible, and a third party (B). This is called 
“answerability’”’( Duff, 2005; Duff, 2009; Duff, 2019). 


As far as liability is concerned, we are referring exclusively to 
the situation of a person who performs a certain action in 
stipulating a specific contract, i.e. a legal obligation and its 
related legal consequences arising from it. Also in this 


problematic Duff stated ex novo that: 


“(...) we can have (criminal) responsibility without necessarily having 
(criminal) liability. Consider, for example, the case of a crime committed in 
the absence of illegality (...)” (Duff, 2005). 

In the European context, to avoid technical confusion, especially 


for jurists, the word responsibility was preferred in a unique 
way!?. 
Accountability comes in various dimensional forms since 
criminal liability is one of them (Heyns, 2016). Accountability 
is thus founded in the traditional way on a notion that includes 
the relative control. Control that has to do with constituent 
12£European Parliament, Resolution on artificial intelligence: questions relating to 
the interpretation and application of international law as far as the EU is concerned in 
relation to civilian and military uses and state authority outside the area of criminal 


justice, (2020/2013(IND), recital I, artt. 14, 24, 25, 49, 52, 70, 72: 
https://www.europarl.europa.eu/doceo/document/TA-9-2021-0009_EN.html 
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elements of the crime such as a dominable conduct of the 
perpetrator (Keiler, 2013) and in a causal link with criminal 
liability (Ashworth, 2009; Blomsma, 2012; Keiler, 2013; Keiler, 
Roef, 2019). Criminal liability is imposed on those who are 
aware of their actions as well as the implications of their actions 
or inactions in such a way as to be able to affirm that they have 
chosen a certain behavior (Ashworth, 2009). And how do all 
these first affirmations of responsibility enter, work and bring 
consequences to an algorithmic action? 

Finally, trying to analyze the relationship of responsibility and 
the linguistic nuances it uses, it is understood that the 
relationship between AI and criminal law is based on a 
relationship between conflict and collaboration (Vuletic¢, 
PetraSevic, 2020). The conflict zone leads back to the 
substantive criminal law which directs a human subject as 


responsible. 


The Resolution of the European Parliament “Artificial 
intelligence in criminal law” and the Artificial Intelligence 
Act 

On 6 October 2021, The European Parliament approved a 
Resolution entitled: “Artificial intelligence in criminal law and 


its use by police and judicial authorities in the criminal field”). 


13European Parliament resolution of 2021 on artificial intelligence in criminal 
law and its use by police and judicial authorities in criminal matters 
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This is not a resolution of a binding nature but a starting point 
which develops the related considerations that guide the EU to 
the related profiles of criminal liability that are connected with 
the AI. In European criminal law also there is not yet a general 
part where we can include the related systems of the AI. This 
gap, aS we can see later, generates transversal and dogmatic 
problems (Klip, 2021). 

However, in art. 13 the Resolution just mentioned included the 
notion of legal responsibility and liability as a potential 
prejudicial effect of the AI systems that are used in the context 
of criminal justice. According to art. 13. the European 
Parliament also extends this resolution to other areas: 


“(...) where the use of AI is growing, and connected to the complexity of the 
development and functioning of these technologies (...). The creation of an 
AI system requires the contribution of multiple people, organizations, 
mechanical components, algorithms, software and human users in often 
complex and problematic environments (...)”!4. 

Thus, it is important to establish a mechanism of an attributive 


nature as referred to in art. 13. The recital J also states that: 


“(...) a clear model is needed for attributing responsibility for the potential 
harmful effects of AI systems in the field of criminal law (...)”!. 


(2020/2016(INI)). 


14European Parliament resolution of 2021 on artificial intelligence in criminal 
law and its use by police and judicial authorities in criminal matters 
(2020/2016(IND), recital 1. 


15European Parliament resolution of 2021 on artificial intelligence in criminal 
law and its use by police and judicial authorities in criminal matters 
(2020/2016(IND): “J. whereas a clear model for assigning legal responsibility for the 
potential harmful effects of AI systems in the field of criminal law is imperative; 
whereas regulatory provisions in this field should always maintain human 
accountability and must aim, first and foremost, to avoid causing any harmful effects 
to begin (...)”. 
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Regulatory rules in this area should always uphold human 
responsibility and their first and foremost aim must above all be 
the prevention of any adverse effects!°. 

Regulatory provisions always guarantee “human accountability” 
and so the word responsibility or liability is not used in them, 
given that they have a different basis that intersects with an 
offense which does not necessarily include a legal liability. The 


recital J we can say that it is in contrast with art. 13 that: 


“(...) takes note of the challenges relating to the correct identification of legal 
responsibility and liability for potential damages, given the complexity of the 
development and functioning of AI systems; considers it necessary to set up a 
clear and fair regime for attributing legal responsibility and accountability for 
the potential negative consequences produced by such advanced digital 
technologies; stresses, however, that the first and foremost aim must above 
all be the prevention of such consequences (...)”!”. 


Here we see that the notions of legal responsibility and liability 
are used in the words of legal responsibility and imputability. 


The use of these words shows the effort of the European 


16European Parliament resolution of 2021 on artificial intelligence in criminal 
law and its use by police and judicial authorities in criminal matters 
(2020/2016(INJ)), recital J. 


17European Parliament resolution of 2021 on artificial intelligence in criminal 
law and its use by police and judicial authorities in criminal matters 
(2020/2016(IN]), art. 13: “Recognises the challenges to the correct location of legal 
responsibility and liability for potential harm, given the complexity of development 
and operation of AI systems; considers it necessary to create a clear and fair regime 
for assigning legal responsibility and liability for the potential adverse consequences 
produced by these advanced digital technologies; underlines, however, that the aim 
must, first and foremost, be to prevent any such consequences materialising to begin 
with; calls, therefore, for the application of the precautionary principle in all 
applications of AI in the context of law enforcement; underlines that legal 
responsibility and liability must always rest with a natural or legal person, who must 
always be identified for decisions taken with the support of AI; emphasises, 
therefore, the need to ensure the transparency of the corporate structures that 
produce and manage AI systems (...)”. 
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Parliament to put more precisely the type of responsibility that 
the legal consequences derive from. Thus the liability as well as 
the legal responsibility carry legal responsibilities as we read in 
the recital J and the term liability means, in our opinion, 
imputability which in criminal law represents a different 
meaning from that of liability. 

In the recital J the guarantee of a form of human accountability, 
rectius responsibility is discussed. Instead art. 13 establishes the 
relative obligation for both a natural and legal person to be 
legally responsible and liable for the relative decisions that are 
taken with the support of a system of the AI. It is a matter of an 
obligatory figure of the accountable human or of a “legally 
responsible and liable human” who bring into play the 
accountability guaranteed with non-criminal instruments, of 
auditing procedures which always guarantee an identified 
criminal subject who is responsible in force of the position of 
guarantee where one's subjectibility to the type of crime is 
provided for in the individual legal systems, and with the 
relative difficulties that apply and that derive from the same 
recitals that we analyze. 

The European Parliament Resolution through art. 13 seeks to 
consistently apply the precautionary principle for all related 
applications of the AI that enter into law enforcement, as it 


underlines the legal responsibility and imputability that falls on 
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natural or legal persons who identify the decisions taken with 
the support of the AI, thus emphasizing the security and 
transparency of the companies that produce and manage AI 
systems!* Thus the first and main objective of a prevention with 
negative effects that is inspired by the precautionary principle is 
framed. The invocation of this principle immediately makes us 
think of the criminal law of the risk that is proposed by the 
European regulation presented by the European Commission 
itself during April of 2021 entitled as: “AI Act’’!’. 

The AI Act is based on a risk-based approach. The regulation 
establishes a few different levels of risk which are concentrated 
on the used positions of: unacceptable, high risk, low or minimal 
risk. Each level of risk also corresponds to the obligations and 
requirements of the high-risk system, arriving at what interests 
us, namely human oversight. Human intervention provides for 
the measure to prevent, i.e. govern a risk that occurs with 
detrimental effects that are caused by an AI system. The human 
supervisor will be one of the responsible subjects where this risk 
materializes and is in default of his/her surveillance obligations, 


i.e. of the relative subjects who are “legally responsible and 


18European Parliament resolution of 2021 on artificial intelligence in criminal 
law and its use by police and judicial authorities in criminal matters 
(2020/2016(IND), art. 13. 

19European Commission, Proposal for a regulation of the European parliament 
and of the council establishing harmonized rules on artificial intelligence (law on 
artificial intelligence) and amending certain legislative acts of the union, 
2021/0106(COD), 21 April 2021. 
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liable as per the aforementioned article 13”. 
The AI Act through the recital 48 seeks to affirm the related 
human surveillance measures that are adequate to guarantee that 


the system: 


“(...) responds to the human operator, and that the natural persons who have 
been entrusted with human surveillance have the skills, training and authority 
necessary to carry out this role (...)’°. 

Continuing with article 14 of the AI Act which states: 


“(...) to prevent or minimize the risks to health, safety or fundamental rights 
which may arise when a high-risk AI system is used in accordance with its 
intended purpose or under conditions of reasonably foreseeable misuse 
(...)724, 

Paragraph 4 includes: 


“(...) a detailed list of the actions that should be implemented by those 
entrusted with human surveillance. These include, for example, the 
functioning of the high-risk AI system or its interruption by means of a 
“stop” button or similar procedure (Art. 14, c. 4, lett. e) (...)”. 

Despite all this European effort we do not yet have a binding 


force in our hands which represents an evolutionary clue not so 
much in European criminal law as in the creation of a regulatory 
framework which starts from the basis of carrying out the 
related reflections which take shape as challenges for the 
evolution of the EU law and above all the recognition of 


individual conduct of human oversight?” and the attribution of 


20European Commission, Proposal for a regulation of the European 
parliament and of the council establishing harmonized rules on artificial 
intelligence (law on artificial intelligence) and amending certain legislative acts of 
the union, op. cit. 

21European Commission, Proposal for a regulation of the European parliament 
and of the council establishing harmonized rules on artificial intelligence (law on 
artificial intelligence) and amending certain legislative acts of the union, op. cit., art. 
14, lett. c). 

22https://ec.europa.eu/futurium/en/ai-alliance-consultation/guidelines/1.html 
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responsibility for fault. 


Human oversight and/or human-in-the loop 

Autonomous work in the AI systems results in the 
malfunctioning of the mechanisms for attributing criminal 
responsibility. The AI Act offered a first solution on it as it was 
based on the notion of human surveillance using the so-called 
human-in-the loop term (HITL) (Enarsson, Engqvist, Naarttijarvi, 
2022). It is a realization of the AI systems where the output 
model is developed with the interaction of a human agent who 
plays the role of the teacher in the training phase of the system 
thus providing feedback to the machine which obtains the result 
(Wu, 2021). 

For some, the connection of the human being in the loop of AI: 


“(...) could seem prima facie a reassuring solution, based on a circular logic 
that is distracted by the intrinsically harmful uses of automated systems. 
Policymakers are turning to humans to mitigate the risks posed by AI systems 
based on the assumption that humans are actually able to oversee their 
decision-making processes (...) the concept of human oversight seems to be 
used as a sort of miraculous ointment, to remedy the potential risks posed by 
algorithmic action. The real risk, however, is that of making the man who 
supervises a scapegoat, in clear contrast with the general principles of our 
criminal law, in particular with the refusal of strict liability (...)” (Green, 
Kaka, 2021). 

Of course there are many questions. Already in the field of AI 


human oversight and accountability continue to proceed closely. 
23European Commission, Proposal for a regulation of the European parliament 
and of the council establishing harmonized rules on artificial intelligence (law on 


artificial intelligence) and amending certain legislative acts of the union, op. cit., art. 
14. 
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The concepts of supervision do not allow us to immediately 
arrive at human oversight or criminal liability. Why? Why does 
human surveillance work as a position holder which is secured 
by a human or a machine? And on the other hand, the European 
discipline on human oversight is still in an embryonic stage and 


the precautionary rules of conduct have not yet been established. 


Conflicts and warranty positions 

As can be understood, it is obvious that a position of guarantee 
is needed for the person who controls the AI system. This 
control includes protection and also the distinction between non- 
compliance with a guarantee obligation and non-compliance 
with a duty of care. We can identify the potential warranty 
position and the semi-autonomous driving that a driver carries 
out inside a vehicle. This is not a simple vehicle like a car but a 
much more complicated machine that activates the driving 
system of an intervention in the driver who takes control of the 
car (Banks, 2014). 

The vehicle inspector does not mean that has the full 
responsibility in the event of the omission of a harmful event. In 
our case, the inspector who puts the vehicle into circulation has 
the risk of the demande de reprise”. The transfer of control of 
rena “(...) hands and feet free but not “mind free” driving 


25See: Ordonnance n° 2021-443 du 14 avril 2021 relative au régime de 
responsabilité pénale applicable en cas de circulation d'un véhicule a délégation de 
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the activity of a risky nature that performs to the vehicle results 
in the liability of a driver who steers poorly and bears the 
associated risks. The problem and the question is the reference 
to the case of an omissive liability where the liability has to do 
with a culpable permissive conduct when the control activity can 
fall within the scope of application of the ordinary rules that 
have to do with the circulation. The problem is legality since 
European sources and internal legislation do not expressly 
provide for anything and given the interpretative possibility of 
AI systems within the application sphere of the related legal 
guarantee obligations that are already existing based on the 
related technologies and the effort towards this direction which 
may be suitable for identifying the guarantor who is invested the 
obligation and holder of an effective power of impediment 
according to the founding principles of criminal law. The 
conception, production, marketing and use of the systems of the 
AI in the course of this process, i.e. many hands problems, lead 
to the relative responsibility. An all-encompassing position 
entails the identified obligation of the human overseer that 
foresees any damage that is caused by an AI system that is 
contrary to the principle of taxability. 

And in the case of omissive liability, the subjective element 


meets with the problems that the AI systems have raised with 


conduite et 4 ses conditions d'utilisation: 
https://www.legifrance.gouv.fr/loda/id/JORFTEXT000043370894/2023-03-03 


Yearbook of International & European Criminal and Procedural Law, vol. 2. ISSN:2732-9917 


356 


the ascertainment of the causal link and the profile of the 
impossibility of preventing the event. The explanation linked to 
the possibility for humans to dominate a certain event and the AI 
system simply leads to a set of inputs and outputs that is not 
only difficult to explain but also to answer. Which inputs lead to 
the related outputs? What is human behavior and _ its 
consequences of a harmful nature that increases in tempis? The 
action of AI systems collides with the simultaneous presence of 
both human and not alternative causal factors (Hildebrandt, 


2014; Santoni De Sio, Mecacci, 2021)°. 


Conflicts and the subjective element of crime 
Speaking of the conception of the AI system as a culprit we 


must take into consideration that: 


“(...) the possibility of identifying a real culpability-and not a simulacrum of 
the same-in the hands of an AI system raises (...) not few difficulties, logical 
and ontological (...)”. 

Some scientists who deal with robotics (Pagallo, 2018) speak of 


liberation or rather of the words of the Chopra and White 


26According to Santoni de Sio and Mecacci: “(...) a vehicle may be operated by a 
driver D1, with the assistance of the automated driving system AS, produced by the car 
manufacturer X, powered with digital systems developed by the company Y, possibly 
including some form of machine learning developed by the company Z, and enriched by 
data coming from different sources, including the driving experience of drivers D2, D3 (...) 
Dn; vehicles in this system are in principle subject to a standardisation process done by the 
agency S, the traffic is regulated by the governmental agency G, drivers are trained and 
licensed by the agency L etc. Second, some specific features of present-day learning AI 
systems may make this interaction particularly unpredictable-typically when the vehicles’ 
performance is potentially redesigned by the second on the basis of new data acquisition and 
processing-and opaque, if the reasoning scheme underlying systems’ actions is not easily 
accessible to their controllers, regulators, or even their designers (...)”. 
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(Chopra, White, 2011): 


“(...) robots will be capable of sensitivity to the command of the criminal law 
and will therefore be reproachable by punishment (...). We cannot speak of 
robotic guilt, because AI systems lack self-awareness (that is, they are not 
aware of being conscious), free will and moral autonomy (...)”. 


A second path is found and related to human responsibility 
involving the human behind the machine. Why are we interested 
in this trend? Because it has to do with the negligent charge, as 
an area of permitted risk that makes us feel the health and 
transport sector in national criminal law. Linking to the system 
of the AI we must also have the relative technical-scientific 
knowledge for the application of the rules of conduct that persist 
in a specific way to the new cases that these systems cause and 
develop again from time to time. The invocation of the 
precautionary principle by the European authorities is adequate 
and in a general way this principle serves the choices of 
legislative policy rather than those of penalisation. Thus we get 
to exceed the thresholds of requests that have to do with the 
adoption of new rules of a precautionary nature thanks to 
progress in scientific laws that enter into the reconstruction and 
explanation of AI systems that are more complex. Creating a 
programming fault drops responsibility for damages that are 
committed by the creator's own creature. Who is to blame at this 
point? How can we talk about precautionary rules and the 
related conduct of the implementation of an AI system? And is 


the conduct of the human being? And who is the human agent 
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who bears the responsibility? 

Our attention falls on the potential controller who autonomously 
or semi-autonomously uses an AlI-based system. Let's talk about 
automation complacency (Giannini, Kwik, 2023) which is 
connected with plane crashes and the phenomenon of task 
automation which leads the human supervisor to trust that the 
machine that takes care of it effectively is the consequence to 
stop continuing attention (Wiener, 1981; Parasuraman & 
Manzey, 2010; Smiley, 2022). And in a second way where the 
automation bias (Zavrsnik, 2020), obvious the tendency of the 
human being who brings excessive trust in the relative 
recommendations that are the products of a computer system as 
an amplified phenomenon that the AI reduce the attentions of 
the human agent struggling with the machine as they also 
decrease the threshold of the enforceability of a diligent conduct 
on the part of the same. 

We can think that the supervisor is the one who examines the 
sources and notes how attention is given to the roles covered in 
the development chain of an AI system. On the other hand, the 
reference to the figure of the programmer becomes an epicenter 
of criminal liability. The AI team includes the various figures 
including the data analyst and the data scientists who deal with 
the interpretation of data in a pertinent and exhaustive way. Data 


and machine learning engineers build a machine learning fabric 
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and its programmers turn the model into code. The roles are 
cumulative with that of the user where the subject who operates 
the system of the AI corresponds, is equivalent to the type of 
responsible model agent. 

Of course there is also the issue of criminal liability which is 
connected with the errors and inadequacy of the training data of 
the AI system. Machine learning based systems are so fast and 
hungry for new processing data. The more foods you pass, the 
more tagged and accurate the data is for your prediction. The 
aim of course is the classification and provision of possible 
types of classifications of data such as databases that are riddled 
with errors (Kang and others, 2022). Systems of the AI are 
databases that are flawed and are used for critical environments 
where wrong generalization is fatal. We recall the Uber case 
(Wakabayashi, 2018), as an accident that was caused by a 
problem: 


“(...) of inability of the vehicle software to identify the presence of a 
pedestrian crossing on the road without using the pedestrian crossing (...)”7. 


What if the algorithm was trained on the bad data after 
downloading bad or distorted data? In this case who is 
responsible? Data collection and generation, labeling and 


evaluation of one's own qualities are essential to avoid those 


27‘The system never classified her as a pedestrian-or correctly predicted her 
path-because she was crossing N. Mill Avenue at a location without a crosswalk, 
and the system design did not include consideration for jaywalking pedestrians”. 
National Transportation Safety Board, Collision Between Vehicle Controlled by 
Developmental Automated Driving System and Pedestrian, Tempe, Arizona, March 18, 
2018, Highway Accident Report NTSB/HAR-19/03, 16. 
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who make mistakes, as well as the verification of harmful 
events. On the other hand, perhaps they have not sufficiently 
gained the relative popularity that has to do with the analysis of 
a criminal nature at national and European level. Within that 
context we think of the “Data Centric AI’ (DCAI) movement, 
which focuses on the engineering of machine learning and the 
related modeling of the data that underlies and uses it for model 
training and evaluation. 

The error of the supervisor and the damage caused by an AI 
system lead back to a malfunction that raises problems for the 
system, i.e. not only liability for damage from a defective 
product but also an error committed which, during the training 
of the algorithm, may be in same line as a mistake having to do 
with assembling a car. However in short we can say that the 
system of the AI as a result of the cooperation of a still very 


complex model. 


Concluding remarks 

Talking about criminal liability in the AI system is not a new 
and even exhaustive topic. From time to time and also in films 
they speak of evil robots where human responsibility has to do 
with the control of machines that are going crazy and act in a 
crazy and uncontrollable way (Floridi, 2016; Ellul, 2022). Of 


28https://datacentricai.org 
29Think instead of the false Maria in Metropolis (1927); Hal 9000 in 2001: A 
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course, as we have seen in the European sector, the purpose is 
the regulation of the illicit conduct of the AI (Lehman-Wilzing, 
1981; Chesterman, 2021). The reflections are many and not 
exhausted. The profiles of criminal liability remain the same, i.e. 
causal link, willful misconduct, fault. The key for the player in 
this sector is the global examination of a complex problem 
which is limited to the analysis of the so-called conflict zones 
where the possible connection profiles have to do with the 
concept of human oversight and the criminal liability of guilt. 
The diagnosis of responsibility remains ambiguous and until 
now European criminal law has not helped to clarify it by 
providing for example elements of monitoring and evaluation 
that help the national legislator to better identify the responsible 


person and to protect and evolve the technology for human use. 


Space Odyssey (1968)(...); C3PO in Star Wars (1977); Rachael in Blade Runner (1982); 
Data in Star Trek: The Next Generation (1987); Agent Smith in The Matrix (1999) or 
the disembodied Samantha in Her (2013). 
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